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EXAMPLE 30 

INTELLIGENT INTERNET APPLIANCE 

A further application of the present technologies is in a so-called "Internet 
appliance". These devices typically are electronic devices which have a concrete 
function (i.e., do more than merely act as a generic server) and typically employ at least 
as a secondary interface, a web browser 3205 . In addition, these devices provide a 
TCP/IP network connection and act as a web server, usually for a limited type of data. 
Therefore, in addition to any real human interface on the device, a web browser 3205 
may be used as a virtual interface 3304. 

According to the present invention, such an Internet Appliance is provided 
according to the present invention with advanced features, for example adaptivity to the 
user, to the environment, or intelligent algorithms which leam. In fact, a preferred 
embodiment provides 3301 a rather generic device which serves as a bridge between the 
Internet, a public packet switched network 3202 which employs TCP/IP, and a local area 
network 3213, for example in a residential, industrial or office environment. The device 
may further abstract the interface functions for a variety of other devices 3212 as nodes 
on either the Internet or local area network 3213, to provide a common control system 
and interface. 

A preferred embodiment also encompasses certain other features which may be 
used as resources for the networked devices or as usable features of the device. 

The Internet, or other wide area network, may be connected in any known 
manner, for example, X.25/ISDN D-channel, dial-up over POTS (e.g., v.34, v.90, v.91), 
ISDN, xDSL, ADSL, cable modem, fi-ame relay, Tl line, ATM, or other communications 
system. Typically, a system is provided with either a commonly used access method, 
such as v.90 or ISDN, or a replaceable communications module with a generic interface. 
Such systems are well known. 

The local area network 3213 is also well known, and may include, for example, as 
a physical layer, 10 Base T, 100 Base T, HomeRun (Cat. 3 twisted pair/telephone twisted 
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pair/power line transmission, from Intel Corp., e.g., Intel 21 145 device/Tut systems), 
Universal Serial Bus (USB), Firewire (IEEE- 1394), optical fiber, or other known 
computer network. The protocol may be, for example, TCP/IP, IPX, ATM, USB, lEEE- 
1394, or other known or proprietary appropriate communications protocol. 

While not required, a particular aspect of a preferred embodiment according to the 
present invention is the ability to interface "dumb" devices as nodes on the LAN 3213 
with an intelligent device 3201. while allowing the user to interact primarily with the 
intelligent device 3201 . This scheme therefore reduces redundancy and increases 
functionality. 

Therefore, in an exemplary embodiment, an intelligent home is established, with 
most or all electrical appliances 3223 and electronic devices interfaced with the system, 
for example through the aforementioned Homerun system, using any of the supported 
physical layers. Each device is provided as a relatively simple control, for example, 
remotely controllable (or where applicable, dimmable) lights 3224. control over normal 
use and peak electrical demand of heavy appliances 3223. as well as inter-device 
communications for consumer electronics 3221, Therefore, the intelligent device acts as 
an external communications and control node for the entire network, and may, for 
example, control telephony 3214 functions in addition. 

Exemplary devices to be controlled in a home include household appliances 3223. 
HVAC 3215 . alarm systems 3217 . consumer electronics 3221 . and the like, and/or 
provide for communications purposes. An alarm system 3217 embodiment, for example, 
may employ a video camera input 3219 for capture and analysis of images, as well as 
motion or irregularity detection. The intelligent device 3201 may, for example, employ 
neural networks or other intelligent analysis technology for analyzing data pattems 
indicative of particular states. An alarm output may be produced, for example, through 
standard alarms, as well as through a telephone 3214 interface of the system. 

The system may therefore set/control/monitor the status of any home-based 
device - oven, stove, alarm, washing machine, dryer, iron, lights, computer, oil/gas 
bumer, thermostat 3222. location of automobiles 3218 . camera, pump 3226 (pool, sump), 
sprinkler 3225 , stereo/video systems, home surveillance system 3216 . This may be 
especially important if the user is away from home for an extended period of time, or if 
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he or she wants to change the schedule of something, or travel plans change. For a home 
surveillance system 3216 , pattern recognition may be employed to monitor all sensors, 
including cameras, to detect abnormal patterns or changes in condition. 

Thus, since the intelligent device incorporates a web server, the physical 
proximity of the user is not critical for interaction with the device, and all devices on the 
LAN 3213 may be controlled remotely, automatically, and in synchrony. 

In one embodiment, the intelligent device includes a videoconferencing 
3220/ video capture system, including any or all known features for such systems, for 
example as described in the backgroimd of the invention. Therefore, in addition to a base 
level of functionality, such an embodiment would also likely include (a) telephony 3214 
interface, (b) video capture, (c) video codec, (d) audio capture, (e) audio codec, (f) full 
duplex speakerphone, (g) video output, and (h) audio output. 

In another embodiment, a speech interface is provided for interpreting human 
speech as an input and/or producing synthesized speech as an output. Therefore, such a 
device would include speech recognition and/or synthesis technologies, as well as a 
semantic data processor. 

[ Pi ' cfcrabl c ] Preferably, the device allows use of a simplified web browser 
interface 3205 . such as which may be supported by personal digital assistants (PDAs) and 
enhanced digital data cellular telephones, e.g., handheld device markup language 
(HDML). This, for example, allows a remote user to communicate through wireless 
networks 3211 or the like, and therefore avoids the need for a fiiU personal computer as a 
human interface. 

Advantageously, the device may be interfaced with a telephone 3214 
communication system, allowing use as a voice and/or video message recorder, and 
allowing remote access to the stored information, either through a dialup connection 
and/or through the network. In this case, the intelligent device 3201 may act as a 
computer telephony mterface, and all communications devices logically under this device 
act as "net phones", i.e., voice communications devices which commvmicate over data 
networks. Therefore, all telephony control and computer telephony functions may be 
integrated into the device, for example, voice mail, auto-attendant, call center, and the 
like. Further, the Internet interface allows remote messaging and control over the 
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telephony system, as well as virtual networking, Internet telephony, paging functions, and 
voice and data integration. 

The intelligent device 3201 may also interface with various media electronics 
devices, and for example, may act as a "rights server" 3208 or other aspect of a copyright 
protection and royalty collection/enforcement system 3307 . Typically, these functions 
entail e-commerce functions, and may require X.22 and/or XML communications and 
translations. In addition, such functions also typically involve encryption/decryption 
3207, as well as key management, which are also preferably supported by the device. 
Such support may be in hardware or software. 

Another aspect of the invention provides an index and/or catalog database 3204 
for media information 3209 or media metadata 3210 information. Thus, data relating to a 
VCR tape or other recorded media may be subjected to search criteria without requiring 
access or contemporaneous analysis of the media content itself. Therefore, a preferred 
embodiment of the intelligent device includes mass storage and retrieval capability 3204, 
for example, magnetic disk, RW-CD, or RW-DVD. This mass storage and retrieval 
capability 3204 may be used, not only for databases, but also for computer software, 
media and content storage and retrieval 3303 . Thus, the device may also serve as a video 
data recorder, capturing video data and storing it digitally, for example, employing the 
aforementioned video and audio codecs. In this case, it is preferable that the intelligent 
device 3201 also include a direct media access port 3203 , for example a broadcast TV 
tuner, ATSC/HDTV tuner, cable tuner, DVD reader, CD reader, satellite video decoder, 
NTSC composite/S-VHS, and/or other type of media content information input 3302. 
With such storage, the intelligent device 3201 may also assume the standard functions of 
computer network servers, for example, file serving, print serving, fax serving, 
application serving, client/server application support, as well as traditional networking 
functions, such as bridging, routing, switching, virtual private network, voice-over-IP, 
firewall functions, remote access serving, and the like. It should also be apparent that the 
intelligent device 3201 may also serve as a personal computer 3206 itself, and thus does 
not require additional systems for basic functionality. 

In a media recording system embodiment, the system preferably notifies the user 
if the "program", i.e., instructions, are incomplete, ambiguous, or impossible to complete. 
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For example, if a single channel selector is provided, no more than one channel may be 
monitored at a time. Further, where irreversible actions are necessary, the user is 
preferably informed and allowed to make a choice, for example, if lack of storage space 
forces a choice to be made between new and archival material. A conflict management 
system is provided which arbitrates between the conflicting demands, for example if a 
second user is programming the same device (for example, the VCR) to record a show at 
the same time^. 

Thus, it is apparent that the intelligent device 3201 according to this embodiment 
of the present invention may incorporate many different functions, some of which are 
defined purely by software and processing availability, and others by particular hardware 
devices for performing specific functions. 

Another aspect of the invention defines a special training mode of the intelligent 
device, which allows the user to improve the functionality of the system by ensuring that 
any intelligence algorithms will correctly operate in an anticipated and/or desired maimer. 
In this mode, responses of the user are provoked which indicate user preferences, 
preferably in a manner which resolves ambiguities encountered with prior data sets. 
Thus, where the system identifies a situation where a decision is difFicuh, e.g., where the 
data analysis does not output any selected actions which will likely correspond to the user 
desires or preferences, or where ex post facto the user indicates that an inappropriate 
choice was made, the particular data structures may be stored and abstracted for later 
presentation to the user. In this case, such structures are presented by the system to the 
user, during a training session, to train the system relating to the desired response to 
particular data environments. In this way, the user is not necessarily burdened with 
training tasks during normal use of the device, and opportunities for such training are not 
lost. Where the system is untrained, and an "intelligent" response or mode of operation 
cannot be resolved, a default mode of operation may be defined. Further, such a default 
mode is preferably always available, at the request of the user, thus allowing use where 
an adaptive system is undesired or difficult to employ. 

In a television application, the Internet appliance preferably has access to an 
electronic program guide (EPG). Such EPG systems are known, and typically provide an 
efficient staring point for user programming. These EPG may be provided as an 
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embedded signal in a broadcast stream, through a dial-up network, through the internet, 
or on distribution media, such as CD-ROM, OCR scanning of TV-Guide (or the like) or 
other known means. EPGs contain a concise semantic description of program content, 
which typically is both sufficient for user evaluation, and brief enough for rapid 
evaluation. The system may therefore analyze user preferences in this semantic space 
and provide adaptive presentation of elements of the EPG to the user. Of course, a media 
data stream analysis embodiment of the invention, as disclosed above, may be used in 
conjvmction with or in lieu of the EPG system. 

The system preferably maintains an updated index of available data. Thus, newly 
acquired data is added to the index, and deleted data is purged from the index. The 
system preferably compares new data to previously encountered data, to avoid redundant 
processing. For example, the system preferably recognizes events/programs that have 
previously been recorded, and checks to determine whether they are still in the index. In 
this context, the user is preferably provided with low-level file maintenance tools, for 
example to manually control the addition or deletion of data, which is then correctly 
represented in the index. 

Because the Internet appliance is connected to the Internet, so-called multicasts 
may be monitored for correspondence with user preferences. Therefore, it is understood 
that the operation of the present invention is not limited to traditional television 
broadcasts, and that streaming video and audio, as well as stored images, sound files 
(e.g., MIDI, MPS, A2B, RealAudio), text, and multimedia streams may be analyzed 
based on the adaptive principles presented herein 3305 . 

The system may also integrate Internet data with other types of data, for example 
providing access to stored or static data corresponding to a data stream. The retrieval and 
storage of such data may also be adaptively controlled in accordance with the present 
invention. Thus, it is expressly understood that the intelligent device may act as a "VCR" 
(albeit not necessarily employing a known type of videocassette tape), to record media 
3306 , 

The Intemet appliance may also operate autonomously, capturing data which 
corresponds to user preferences and profiles, thus reducing latency for the user, and 
potentially shifting data transfers to off-peak periods. Such a system operates in this 
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mode as a so-called "agent" system. Likewise, the device may also be linked to other 
intelligent devices, to provide an intelligent interaction therebetween. 

The preferred user interface maintains user levels constant over long periods, i.e., 
not rapidly adaptive, to allow for quick accessing over a low bandwidth connection, such 
as a telephone, or using succinct displays, such as might be found on a personal digital 
assistant. Thus, the user can rely on memory of the interface functionality and layout to 
reduce data transmissions and reduce search time. In one embodiment, the interface may 
be "forced" to a particular type, as either a permanent interface, or as a starting point for 
adaptivity. Thus, the user may be provided with an interface design mode of operation. 

The user interaction with each "device", which may be real or virtual 
(implemented as a software construct in a relatively general purpose computer), is 
preferably carefully designed for each device. A common user interface paradigm is 
preferably provided for corresponding functions, while the user interface is preferably 
optimized for dealing with the specific functions of each particular device. Thus, a 
similar user interface and screen layout is employed for functions that are the same across 
a variety of devices. In this regard, it is an aspect of an embodiment of the invention to 
translate user interface systems, even in a high level state, to other forms. Thus, in a 
multi-brand environment, related components may have native interfaces that are both 
well developed and distinctly different. Therefore, the present invention allows for a 
translation or remapping of the functionality into a common paradigm. Where aspects 
cannot be adequately translated, the native interface may be presented to the user. 
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